feat(scanner): Optimize creation of file archives#11483
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #11483 +/- ##
=========================================
Coverage 58.43% 58.43%
Complexity 1807 1807
=========================================
Files 361 361
Lines 13499 13499
Branches 1383 1383
=========================================
Hits 7888 7888
Misses 5115 5115
Partials 496 496
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
I tested this locally because I'm having a similar problem, but |
|
This PR does not change the downloader, only the Scanner. So, it is expected that the behavior does not change. |
oheger-bosch
force-pushed
the
oheger-bosch/scanner_optimize_file_archives
branch
2 times, most recently
from
650b3c2 to
5b54591
Compare
A file archive is associated with a provenance. When processing provenances with multiple packages, the archive was created repeatedly for each package. Prevent this by grouping the packages by provenance and create only a single archive per provenance. For large repositories containing many submodules, this can significantly reduce the processing time of the scanner. Resolves oss-review-toolkit#11484. Signed-off-by: Oliver Heger <oliver.heger@bosch.com>
oheger-bosch
force-pushed
the
oheger-bosch/scanner_optimize_file_archives
branch
from
5b54591 to
ac9d4be
Compare
3 participants
When processing provenances with multiple packages, for each package, the provenance was downloaded to create the file archive. Prevent this by grouping the packages by provenance and do only a single download. This can have a significant effect for large repositories containing many submodules.